SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation

• URL: http://arxiv.org/abs/2506.03139v1
• Date: Tue, 03 Jun 2025 17:58:57 GMT
• Title: SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation
• Abstract summary: We introduce SVGenius, a comprehensive benchmark comprising 2,377 queries across three progressive dimensions: understanding, editing, and generation.<n>Built on real-world data from 24 application domains with systematic complexity stratification, SVGenius evaluates models through 8 task categories and 18 metrics.
• Score: 46.49501119029113
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) and Multimodal LLMs have shown promising capabilities for SVG processing, yet existing benchmarks suffer from limited real-world coverage, lack of complexity stratification, and fragmented evaluation paradigms. We introduce SVGenius, a comprehensive benchmark comprising 2,377 queries across three progressive dimensions: understanding, editing, and generation. Built on real-world data from 24 application domains with systematic complexity stratification, SVGenius evaluates models through 8 task categories and 18 metrics. We assess 22 mainstream models spanning different scales, architectures, training paradigms, and accessibility levels. Our analysis reveals that while proprietary models significantly outperform open-source counterparts, all models exhibit systematic performance degradation with increasing complexity, indicating fundamental limitations in current approaches; however, reasoning-enhanced training proves more effective than pure scaling for overcoming these limitations, though style transfer remains the most challenging capability across all model types. SVGenius establishes the first systematic evaluation framework for SVG processing, providing crucial insights for developing more capable vector graphics models and advancing automated graphic design applications. Appendix and supplementary materials (including all data and code) are available at https://zju-real.github.io/SVGenius.

Related papers

• Evaluating Constrained Iterative Refinement for Scalable Vector Graphics Generation with Off-the-Shelf VLMs [50.541990269155484]
  We evaluate whether inference-time methods can unlock SVG generation capabilities in off-the-shelf vision-language models (VLMs)<n>Our results highlight both the promise and current limitations of using inference-time methods to adapt general-purpose VLMs for SVG generation.
  arXiv  Detail & Related papers  (2026-08-26T00:51:10Z)
• SVGEval: A Vision-Grounded Framework for Perceptual-Quality Benchmarking and Evaluation in Text-to-SVG Generation [49.00672742747506]
  Multimodal large models are increasingly used to generate scalable vector graphics (SVG)<n>Existing protocols are often code-centric or borrow-image metrics after rendering SVGs.<n>We introduce SVGEval, a vision-grounded multimodal benchmark for human-aligned SVG quality assessment.
  arXiv  Detail & Related papers  (2026-08-03T09:41:18Z)
• VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models [43.3181510471477]
  In practice, original vector source files are frequently lost or inaccessible.<n>We propose VFIG, a family of Vision-Language Models trained for complex and high-fidelity figure-to-SVG conversion.<n> VFIG achieves state-of-the-art performance among open-source models and performs on par with GPT-5.2.
  arXiv  Detail & Related papers  (2026-03-25T17:52:23Z)
• Reliable Reasoning in SVG-LLMs via Multi-Task Multi-Reward Reinforcement Learning [15.21004381065588]
  We present Chain-of-Thought-Reinforcement Learning for SVG, a unified framework to expose the model's reasoning process during SVG generation.<n>By training the SVG model to generate group-level code, our framework significantly improves structural coherence and visual fidelity.<n>Our approach systematically enhances overall generation capabilities, achieving higher task success rates, superior SVG code quality, and exceptional visual fidelity.
  arXiv  Detail & Related papers  (2026-03-17T07:16:30Z)
• IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Framework [20.964700751378547]
  Existing text-to-SVG generation methods do not incorporate visual perception of the final rendered image.<n>We propose an Introspective SVG Generation Framework (IntroSVG)<n>We show that our method achieves state-of-the-art performance across several key evaluation metrics.
  arXiv  Detail & Related papers  (2026-03-10T07:44:51Z)
• Beyond Pixels: Vector-to-Graph Transformation for Reliable Schematic Auditing [34.54168175788343]
  We propose a Vector-to-Graph (V2G) pipeline that converts CAD diagrams into property graphs where nodes represent components and edges encode connectivity.<n>On a diagnostic benchmark of electrical compliance checks, V2G yields large accuracy gains across all error categories, while leading MLLMs remain near chance level.
  arXiv  Detail & Related papers  (2026-02-12T07:50:49Z)
• SVG-T2I: Scaling Up Text-to-Image Latent Diffusion Model Without Variational Autoencoder [90.28800377462197]
  We propose SVG-T2I to support high-quality text-to-image synthesis directly in the VFM feature domain.<n>By leveraging a standard text-to-image diffusion pipeline, SVG-T2I achieves competitive performance, reaching 0.75 on GenEval and 85.78 on DPG-Bench.<n>We fully open-source the project, including the autoencoder and generation model, together with their training, inference, evaluation pipelines, and pre-trained weights, to facilitate further research in representation-driven visual generation.
  arXiv  Detail & Related papers  (2025-12-12T17:45:03Z)
• InternSVG: Towards Unified SVG Tasks with Multimodal Large Language Models [65.49118879021016]
  We present the InternSVG family, an integrated data-benchmark-model suite.<n>At its core is SAgoge, the largest and most comprehensive multimodal dataset for SVG tasks.<n>We propose InternSVG, a unified MLLM for SVG understanding, editing, and generation with SVG-specific special tokens.
  arXiv  Detail & Related papers  (2025-10-13T12:38:04Z)
• SVGen: Interpretable Vector Graphics Generation with Large Language Models [61.62816031675714]
  We introduce SVG-1M, a large-scale dataset of high-quality SVGs paired with natural language descriptions.<n>We create well-aligned Text to SVG training pairs, including a subset with Chain of Thought annotations for enhanced semantic guidance.<n>Based on this dataset, we propose SVGen, an end-to-end model that generates SVG code from natural language inputs.
  arXiv  Detail & Related papers  (2025-08-06T15:00:24Z)
• Rendering-Aware Reinforcement Learning for Vector Graphics Generation [15.547843461605746]
  We introduce RLRF(Reinforcement Learning from Rendering Feedback), an RL method that enhances SVG generation in vision-language models (VLMs)<n>Given an input image, the model generates SVG roll-outs that are rendered and compared to the original image to compute a reward.<n>This visual fidelity feedback guides the model toward producing more accurate, efficient, and semantically coherent SVGs.
  arXiv  Detail & Related papers  (2025-05-27T06:56:00Z)
• Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning [28.291759852111586]
  We introduce a reinforcement learning (RL) based framework that incorporates three core strategies to improve GUI agent performance.<n>With only 3k training samples, our 7B- parameter model achieves state-of-the-art results among similarly sized models.<n> Notably, it attains 47.3% accuracy on the ScreenSpot-Pro dataset, outperforming much larger models, such as UI-TARS-72B, by a margin of 24.2%.
  arXiv  Detail & Related papers  (2025-05-18T11:22:04Z)
• OmniSVG: A Unified Scalable Vector Graphics Generation Model [69.59073636922287]
  We propose OmniSVG, a unified framework that leverages pre-trained Vision-Language Models for end-to-end multimodal SVG generation.<n>By parameterizing SVG commands and coordinates into discrete tokens, OmniSVG decouples structural logic from low-level geometry for efficient training while maintaining the synthesis of complex SVG structure.<n>We introduce MMSVG-2M, a multimodal dataset with two million annotated SVG assets, along with a standardized evaluation protocol for conditional SVG generation tasks.
  arXiv  Detail & Related papers  (2025-04-08T17:59:49Z)
• Leveraging Large Language Models For Scalable Vector Graphics Processing: A Review [0.0]
  Traditional vectorization techniques suffer from long processing times and excessive output complexity.<n>The advent of large language models (LLMs) has opened new possibilities for the generation, editing, and analysis of vector graphics.
  arXiv  Detail & Related papers  (2025-03-06T21:23:17Z)
• RSQ: Learning from Important Tokens Leads to Better Quantized LLMs [65.5558181902098]
  Layer-wise quantization is a key technique for efficiently compressing large models without expensive retraining.<n>We propose RSQ (Rotate, Scale, then Quantize), which applies rotations to the model to mitigate outliers.<n>We demonstrate that RSQ consistently outperforms baseline methods across multiple downstream tasks and three model families.
  arXiv  Detail & Related papers  (2025-03-03T18:46:33Z)
• SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation [81.36747103102459]
  Expressive human pose and shape estimation (EHPS) unifies body, hands, and face motion capture with numerous applications.<n>Current state-of-the-art methods focus on training innovative architectural designs on confined datasets.<n>We investigate the impact of scaling up EHPS towards a family of generalist foundation models.
  arXiv  Detail & Related papers  (2025-01-16T18:59:46Z)
• NeuralSVG: An Implicit Representation for Text-to-Vector Generation [54.4153300455889]
  We propose NeuralSVG, an implicit neural representation for generating vector graphics from text prompts.<n>To encourage a layered structure in the generated SVG, we introduce a dropout-based regularization technique.<n>We demonstrate that NeuralSVG outperforms existing methods in generating structured and flexible SVG.
  arXiv  Detail & Related papers  (2025-01-07T18:50:06Z)
• Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models [14.917583676464266]
  Chat2SVG is a hybrid framework that combines Large Language Models and image diffusion models for text-to-SVG generation. Our system enables intuitive editing through natural language instructions, making professional vector graphics creation accessible to all users.
  arXiv  Detail & Related papers  (2024-11-25T17:31:57Z)
• VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks [60.5257456681402]
  We study the potential for building universal embeddings capable of handling a wide range of downstream tasks.<n>We build a series of VLM2Vec models on SoTA VLMs like Phi-3.5-V, LLaVA-1.6 and evaluate them on MMEB's evaluation split.<n>Our results show that VLM2Vec achieves an absolute average improvement of 10% to 20% over existing multimodal embedding models.
  arXiv  Detail & Related papers  (2024-10-07T16:14:05Z)
• DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation [217.86315551526235]
  We propose a novel hierarchical generative network, called DeepSVG, for complex SVG icons generation and manipulation. Our architecture effectively disentangles high-level shapes from the low-level commands that encode the shape itself. We demonstrate that our network learns to accurately reconstruct diverse vector graphics, and can serve as a powerful animation tool.
  arXiv  Detail & Related papers  (2020-07-22T09:36:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.