NeuralSVG: An Implicit Representation for Text-to-Vector Generation

• URL: http://arxiv.org/abs/2501.03992v1
• Date: Tue, 07 Jan 2025 18:50:06 GMT
• Title: NeuralSVG: An Implicit Representation for Text-to-Vector Generation
• Authors: Sagi Polaczek, Yuval Alaluf, Elad Richardson, Yael Vinker, Daniel Cohen-Or,
• Abstract summary: We propose NeuralSVG, an implicit neural representation for generating vector graphics from text prompts. To encourage a layered structure in the generated SVG, we introduce a dropout-based regularization technique. We demonstrate that NeuralSVG outperforms existing methods in generating structured and flexible SVG.
• Score: 54.4153300455889
• License:
• Abstract: Vector graphics are essential in design, providing artists with a versatile medium for creating resolution-independent and highly editable visual content. Recent advancements in vision-language and diffusion models have fueled interest in text-to-vector graphics generation. However, existing approaches often suffer from over-parameterized outputs or treat the layered structure - a core feature of vector graphics - as a secondary goal, diminishing their practical use. Recognizing the importance of layered SVG representations, we propose NeuralSVG, an implicit neural representation for generating vector graphics from text prompts. Inspired by Neural Radiance Fields (NeRFs), NeuralSVG encodes the entire scene into the weights of a small MLP network, optimized using Score Distillation Sampling (SDS). To encourage a layered structure in the generated SVG, we introduce a dropout-based regularization technique that strengthens the standalone meaning of each shape. We additionally demonstrate that utilizing a neural representation provides an added benefit of inference-time control, enabling users to dynamically adapt the generated SVG based on user-provided inputs, all with a single learned representation. Through extensive qualitative and quantitative evaluations, we demonstrate that NeuralSVG outperforms existing methods in generating structured and flexible SVG.

Related papers

• SVGDreamer++: Advancing Editability and Diversity in Text-Guided SVG Generation [31.76771064173087]
  We propose a novel text-guided vector graphics synthesis method to address limitations of existing methods. We introduce a Hierarchical Image VEctorization (HIVE) framework that operates at the semantic object level. We also present a Vectorized Particle-based Score Distillation (VPSD) approach to improve the diversity of output SVGs.
  arXiv  Detail & Related papers  (2024-11-26T19:13:38Z)
• Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models [14.917583676464266]
  Chat2SVG is a hybrid framework that combines Large Language Models and image diffusion models for text-to-SVG generation. Our system enables intuitive editing through natural language instructions, making professional vector graphics creation accessible to all users.
  arXiv  Detail & Related papers  (2024-11-25T17:31:57Z)
• Vector Grimoire: Codebook-based Shape Generation under Raster Image Supervision [20.325246638505714]
  We introduce GRIMOIRE, a text-guided generative model that learns to map images onto a discrete codebook by reconstructing them as vector shapes. Unlike existing models that require direct supervision from data, GRIMOIRE learns using only image supervision which opens up vector generative modeling to significantly more data.
  arXiv  Detail & Related papers  (2024-10-08T12:41:31Z)
• SuperSVG: Superpixel-based Scalable Vector Graphics Synthesis [66.44553285020066]
  SuperSVG is a superpixel-based vectorization model that achieves fast and high-precision image vectorization. We propose a two-stage self-training framework, where a coarse-stage model is employed to reconstruct the main structure and a refinement-stage model is used for enriching the details. Experiments demonstrate the superior performance of our method in terms of reconstruction accuracy and inference time compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-06-14T07:43:23Z)
• SVGDreamer: Text Guided SVG Generation with Diffusion Model [31.76771064173087]
  We propose a novel text-guided vector graphics synthesis method called SVGDreamer. SIVE process enables decomposition of synthesis into foreground objects and background. VPSD approach addresses issues of shape over-smoothing, color over-saturation, limited diversity, and slow convergence.
  arXiv  Detail & Related papers  (2023-12-27T08:50:01Z)
• Text-Guided Vector Graphics Customization [31.41266632288932]
  We propose a novel pipeline that generates high-quality customized vector graphics based on textual prompts. Our method harnesses the capabilities of large pre-trained text-to-image models. We evaluate our method using multiple metrics from vector-level, image-level and text-level perspectives.
  arXiv  Detail & Related papers  (2023-09-21T17:59:01Z)
• VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models [82.93345261434943]
  We show that a text-conditioned diffusion model trained on pixel representations of images can be used to generate SVG-exportable vector graphics. Inspired by recent text-to-3D work, we learn an SVG consistent with a caption using Score Distillation Sampling. Experiments show greater quality than prior work, and demonstrate a range of styles including pixel art and sketches.
  arXiv  Detail & Related papers  (2022-11-21T10:04:27Z)
• Towards Layer-wise Image Vectorization [57.26058135389497]
  We propose Layerwise Image Vectorization, namely LIVE, to convert images to SVGs and simultaneously maintain its image topology. Live generates compact forms with layer-wise structures that are semantically consistent with human perspective. Live initiates human editable SVGs for both designers and can be used in other applications.
  arXiv  Detail & Related papers  (2022-06-09T17:55:02Z)
• SVG-Net: An SVG-based Trajectory Prediction Model [67.68864911674308]
  Anticipating motions of vehicles in a scene is an essential problem for safe autonomous driving systems. To this end, the comprehension of the scene's infrastructure is often the main clue for predicting future trajectories. Most of the proposed approaches represent the scene with averse averseized format and some of the more recent approaches leverage custom vectorized formats.
  arXiv  Detail & Related papers  (2021-10-07T18:00:08Z)
• DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation [217.86315551526235]
  We propose a novel hierarchical generative network, called DeepSVG, for complex SVG icons generation and manipulation. Our architecture effectively disentangles high-level shapes from the low-level commands that encode the shape itself. We demonstrate that our network learns to accurately reconstruct diverse vector graphics, and can serve as a powerful animation tool.
  arXiv  Detail & Related papers  (2020-07-22T09:36:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.