Compositional SVG Generation via VLM-Driven Hierarchical Semantic Parsing

• URL: http://arxiv.org/abs/2609.14657v1
• Date: Sun, 13 Sep 2026 16:50:07 GMT
• Title: Compositional SVG Generation via VLM-Driven Hierarchical Semantic Parsing
• Abstract summary: We propose a VLM-driven agentic framework for semantic compositional generation.<n>Our pipeline parses visual scenes into semantic and geometric hierarchies via top-down decomposition.<n>Experiments show that our predicted structures surpass the upper bounds of existing flat-generation methods in both grouping quality and editability.
• Score: 18.53393579768945
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While Vision-Language Models (VLMs) excel at visual reasoning, generating structured, editable Scalable Vector Graphics (SVG) remains a fundamental challenge. Existing pipelines predominantly yield flat, semantically agnostic collections of paths, where editing a single object requires manually identifying its constituent paths. To address this, we propose a VLM-driven agentic framework for semantic compositional SVG generation. Our pipeline recursively parses visual scenes into semantic and geometric hierarchies via top-down decomposition, visual grounding, and prompt-driven amodal occlusion recovery, ensuring each component is geometrically complete. Furthermore, we introduce the Semantic SVG Benchmark with human-annotated semantic groups and novel sub-component metrics (Semantic Recall/Precision, PERE) to explicitly evaluate structural compositionality and functional editability. Experiments show that our natively predicted structures surpass the upper bounds of existing flat-generation methods in both grouping quality and editability, while maintaining state-of-the-art visual fidelity.

Related papers

• SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation [62.7676301168359]
  We introduce textbfours, a human-aligned evaluation framework for text-to-SVG generation.<n>We first show that CLIP-based scores barely react to the errors SVG generators actually make, such as wrong colors, counts, and spatial relations.<n>We then introduce a human-annotated dataset for textitSemantic Alignment, measuring how faithfully a generated SVG reflects its caption.
  arXiv  Detail & Related papers  (2026-09-03T13:12:37Z)
• SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models [63.28435103335999]
  We introduce SOCO, a new benchmark for Semantic Object Correspondence.<n>We show that vision foundation backbones encode strong semantic structure but transfer correspondences poorly across related categories.<n>We also show that LVLMs are stronger at text-prompted part localization than at visual-reference cross-image matching.
  arXiv  Detail & Related papers  (2026-05-29T17:58:48Z)
• VAnim: Rendering-Aware Sparse State Modeling for Structure-Preserving Vector Animation [32.226052229379086]
  We present VAnim, the first framework for open-domain text-to-SVG animation.<n>We reconceptualize animation as Sparse State Updates (SSU) on a persistent SVG DOM tree.<n>We also introduce SVGAnim-134k, the first benchmark for vector animation.
  arXiv  Detail & Related papers  (2026-05-02T16:10:55Z)
• Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedback [29.19392406217364]
  We propose a novel generation paradigm that reformulates SVG synthesis as a step-wise, visual-context-aware process.<n>By rendering intermediate code states into a cumulative canvas, the model explicitly observes the evolving visual context at each step.<n>We show that applying this visual loop naively to off-the-shelf models is suboptimal due to their inability to leverage incremental visual-code mappings.
  arXiv  Detail & Related papers  (2026-04-22T16:15:28Z)
• FF3R: Feedforward Feature 3D Reconstruction from Unconstrained views [34.14517479047999]
  This paper introduces FF3R, a fully annotation-free feed-forward framework that unifies geometric and semantic reasoning from unconstrained multi-view image sequences.<n>Unlike previous methods, FF3R does not require camera poses, depth maps, or semantic labels, relying solely on rendering supervision for RGB and feature maps.<n>Two key innovations: (i) a Token-wise Fusion Module that enriches geometry tokens with semantic context via cross-attention, and (ii) a Semantic-Geometry Mutual Boosting mechanism combining geometry-guided feature warping for global consistency with semantic-aware voxelization for local coherence.
  arXiv  Detail & Related papers  (2026-04-10T19:45:24Z)
• Structural Evaluation Metrics for SVG Generation via Leave-One-Out Analysis [7.246398319376824]
  We introduce element-level leave-one-out (LOO) analysis, inspired by the classic jackknife estimator.<n>LOO scores per element through LOO scoring that enable zero-shot artifact detection.<n>We validate these metrics on over 19,000 edits (5 types) across 5 generation systems and 3 complexity tiers.
  arXiv  Detail & Related papers  (2026-04-09T22:50:41Z)
• IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Framework [20.964700751378547]
  Existing text-to-SVG generation methods do not incorporate visual perception of the final rendered image.<n>We propose an Introspective SVG Generation Framework (IntroSVG)<n>We show that our method achieves state-of-the-art performance across several key evaluation metrics.
  arXiv  Detail & Related papers  (2026-03-10T07:44:51Z)
• Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generation [120.23172120151821]
  We propose a novel approach for disentangling visual and semantic features from the backbones of pre-trained diffusion models.<n>We introduce an automated pipeline that constructs image pairs with annotated semantic and visual correspondences.<n>We propose a new metric, Visual Semantic Matching, that quantifies visual inconsistencies in subject-driven image generation.
  arXiv  Detail & Related papers  (2025-09-26T07:11:55Z)
• NeuralSVG: An Implicit Representation for Text-to-Vector Generation [54.4153300455889]
  We propose NeuralSVG, an implicit neural representation for generating vector graphics from text prompts.<n>To encourage a layered structure in the generated SVG, we introduce a dropout-based regularization technique.<n>We demonstrate that NeuralSVG outperforms existing methods in generating structured and flexible SVG.
  arXiv  Detail & Related papers  (2025-01-07T18:50:06Z)
• Open-Vocabulary Octree-Graph for 3D Scene Understanding [54.11828083068082]
  Octree-Graph is a novel scene representation for open-vocabulary 3D scene understanding. An adaptive-octree structure is developed that stores semantics and depicts the occupancy of an object adjustably according to its shape.
  arXiv  Detail & Related papers  (2024-11-25T10:14:10Z)
• Improving Semantic Segmentation via Decoupled Body and Edge Supervision [89.57847958016981]
  Existing semantic segmentation approaches either aim to improve the object's inner consistency by modeling the global context, or refine objects detail along their boundaries by multi-scale feature fusion. In this paper, a new paradigm for semantic segmentation is proposed. Our insight is that appealing performance of semantic segmentation requires textitexplicitly modeling the object textitbody and textitedge, which correspond to the high and low frequency of the image. We show that the proposed framework with various baselines or backbone networks leads to better object inner consistency and object boundaries.
  arXiv  Detail & Related papers  (2020-07-20T12:11:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.