VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models

• URL: http://arxiv.org/abs/2603.24575v1
• Date: Wed, 25 Mar 2026 17:52:23 GMT
• Title: VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models
• Abstract summary: In practice, original vector source files are frequently lost or inaccessible.<n>We propose VFIG, a family of Vision-Language Models trained for complex and high-fidelity figure-to-SVG conversion.<n> VFIG achieves state-of-the-art performance among open-source models and performs on par with GPT-5.2.
• Score: 43.3181510471477
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scalable Vector Graphics (SVG) are an essential format for technical illustration and digital design, offering precise resolution independence and flexible semantic editability. In practice, however, original vector source files are frequently lost or inaccessible, leaving only "flat" rasterized versions (e.g., PNG or JPEG) that are difficult to modify or scale. Manually reconstructing these figures is a prohibitively labor-intensive process, requiring specialized expertise to recover the original geometric intent. To bridge this gap, we propose VFIG, a family of Vision-Language Models trained for complex and high-fidelity figure-to-SVG conversion. While this task is inherently data-driven, existing datasets are typically small-scale and lack the complexity of professional diagrams. We address this by introducing VFIG-DATA, a large-scale dataset of 66K high-quality figure-SVG pairs, curated from a diverse mix of real-world paper figures and procedurally generated diagrams. Recognizing that SVGs are composed of recurring primitives and hierarchical local structures, we introduce a coarse-to-fine training curriculum that begins with supervised fine-tuning (SFT) to learn atomic primitives and transitions to reinforcement learning (RL) refinement to optimize global diagram fidelity, layout consistency, and topological edge cases. Finally, we introduce VFIG-BENCH, a comprehensive evaluation suite with novel metrics designed to measure the structural integrity of complex figures. VFIG achieves state-of-the-art performance among open-source models and performs on par with GPT-5.2, achieving a VLM-Judge score of 0.829 on VFIG-BENCH.

Related papers

• SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation [62.7676301168359]
  We introduce textbfours, a human-aligned evaluation framework for text-to-SVG generation.<n>We first show that CLIP-based scores barely react to the errors SVG generators actually make, such as wrong colors, counts, and spatial relations.<n>We then introduce a human-annotated dataset for textitSemantic Alignment, measuring how faithfully a generated SVG reflects its caption.
  arXiv  Detail & Related papers  (2026-09-03T13:12:37Z)
• GVR-Coder: A Visual-Feedback Framework for Structured SVG Generation in Complex Document and Meeting Scenarios [11.811188580823126]
  We introduce DocMeetSVG-100K, a large-scale SVG dataset tailored for document authoring and meeting review scenarios.<n>At the model level, we propose GVR-Coder, a novel framework designed to generate high-quality logical diagrams from lengthy professional texts.<n>In addition, we introduce reinforcement learning from dual rendering feedback, a mechanism that provides implicit feedback through reward signals to jointly optimize structural complexity and visual aesthetics.
  arXiv  Detail & Related papers  (2026-07-30T11:47:50Z)
• GeoSVG-RL: Geometry-Aware Reinforcement Learning for Layout-Constrained Text-to-SVG Diagram Generation [29.64540884592851]
  We introduce GeoSVG-RL, a specialized reinforcement learning framework designed for layout-constrained text-to-SVG generation.<n>The model first produces a structured layout plan that serves as a geometric contract for the subsequent generation of the SVG code.<n>GeoSVG-RL achieves substantial gains in structural reliability, particularly in arrow-anchor accuracy and text-in-box rates.
  arXiv  Detail & Related papers  (2026-05-25T05:56:44Z)
• IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Framework [20.964700751378547]
  Existing text-to-SVG generation methods do not incorporate visual perception of the final rendered image.<n>We propose an Introspective SVG Generation Framework (IntroSVG)<n>We show that our method achieves state-of-the-art performance across several key evaluation metrics.
  arXiv  Detail & Related papers  (2026-03-10T07:44:51Z)
• SVG-T2I: Scaling Up Text-to-Image Latent Diffusion Model Without Variational Autoencoder [90.28800377462197]
  We propose SVG-T2I to support high-quality text-to-image synthesis directly in the VFM feature domain.<n>By leveraging a standard text-to-image diffusion pipeline, SVG-T2I achieves competitive performance, reaching 0.75 on GenEval and 85.78 on DPG-Bench.<n>We fully open-source the project, including the autoencoder and generation model, together with their training, inference, evaluation pipelines, and pre-trained weights, to facilitate further research in representation-driven visual generation.
  arXiv  Detail & Related papers  (2025-12-12T17:45:03Z)
• InternSVG: Towards Unified SVG Tasks with Multimodal Large Language Models [65.49118879021016]
  We present the InternSVG family, an integrated data-benchmark-model suite.<n>At its core is SAgoge, the largest and most comprehensive multimodal dataset for SVG tasks.<n>We propose InternSVG, a unified MLLM for SVG understanding, editing, and generation with SVG-specific special tokens.
  arXiv  Detail & Related papers  (2025-10-13T12:38:04Z)
• SVGen: Interpretable Vector Graphics Generation with Large Language Models [61.62816031675714]
  We introduce SVG-1M, a large-scale dataset of high-quality SVGs paired with natural language descriptions.<n>We create well-aligned Text to SVG training pairs, including a subset with Chain of Thought annotations for enhanced semantic guidance.<n>Based on this dataset, we propose SVGen, an end-to-end model that generates SVG code from natural language inputs.
  arXiv  Detail & Related papers  (2025-08-06T15:00:24Z)
• SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generation [46.49501119029113]
  We introduce SVGenius, a comprehensive benchmark comprising 2,377 queries across three progressive dimensions: understanding, editing, and generation.<n>Built on real-world data from 24 application domains with systematic complexity stratification, SVGenius evaluates models through 8 task categories and 18 metrics.
  arXiv  Detail & Related papers  (2025-06-03T17:58:57Z)
• NeuralSVG: An Implicit Representation for Text-to-Vector Generation [54.4153300455889]
  We propose NeuralSVG, an implicit neural representation for generating vector graphics from text prompts.<n>To encourage a layered structure in the generated SVG, we introduce a dropout-based regularization technique.<n>We demonstrate that NeuralSVG outperforms existing methods in generating structured and flexible SVG.
  arXiv  Detail & Related papers  (2025-01-07T18:50:06Z)
• Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models [14.917583676464266]
  Chat2SVG is a hybrid framework that combines Large Language Models and image diffusion models for text-to-SVG generation. Our system enables intuitive editing through natural language instructions, making professional vector graphics creation accessible to all users.
  arXiv  Detail & Related papers  (2024-11-25T17:31:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.