Im2Vec: Synthesizing Vector Graphics without Vector Supervision

• URL: http://arxiv.org/abs/2102.02798v1
• Date: Thu, 4 Feb 2021 18:39:45 GMT
• Title: Im2Vec: Synthesizing Vector Graphics without Vector Supervision
• Authors: Pradyumna Reddy, Michael Gharbi, Michal Lukac, Niloy J. Mitra
• Abstract summary: Vector graphics are widely used to represent fonts, logos, digital artworks, and graphic designs. One can alwaysize the input graphic and resort to image-based generative approaches. Current models that require explicit supervision on the vector representation at training time are difficult to obtain. We propose a new neural network that can generate complex vector graphics with varying topologies.
• Score: 31.074606918245298
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vector graphics are widely used to represent fonts, logos, digital artworks, and graphic designs. But, while a vast body of work has focused on generative algorithms for raster images, only a handful of options exists for vector graphics. One can always rasterize the input graphic and resort to image-based generative approaches, but this negates the advantages of the vector representation. The current alternative is to use specialized models that require explicit supervision on the vector graphics representation at training time. This is not ideal because large-scale high quality vector-graphics datasets are difficult to obtain. Furthermore, the vector representation for a given design is not unique, so models that supervise on the vector representation are unnecessarily constrained. Instead, we propose a new neural network that can generate complex vector graphics with varying topologies, and only requires indirect supervision from readily-available raster training images (i.e., with no vector counterparts). To enable this, we use a differentiable rasterization pipeline that renders the generated vector shapes and composites them together onto a raster canvas. We demonstrate our method on a range of datasets, and provide comparison with state-of-the-art SVG-VAE and DeepSVG, both of which require explicit vector graphics supervision. Finally, we also demonstrate our approach on the MNIST dataset, for which no groundtruth vector representation is available. Source code, datasets, and more results are available at http://geometry.cs.ucl.ac.uk/projects/2020/Im2Vec/

Related papers

• Vector Grimoire: Codebook-based Shape Generation under Raster Image Supervision [20.325246638505714]
  We introduce GRIMOIRE, a text-guided generative model that learns to map images onto a discrete codebook by reconstructing them as vector shapes. Unlike existing models that require direct supervision from data, GRIMOIRE learns using only image supervision which opens up vector generative modeling to significantly more data.
  arXiv  Detail & Related papers  (2024-10-08T12:41:31Z)
• SuperSVG: Superpixel-based Scalable Vector Graphics Synthesis [66.44553285020066]
  SuperSVG is a superpixel-based vectorization model that achieves fast and high-precision image vectorization. We propose a two-stage self-training framework, where a coarse-stage model is employed to reconstruct the main structure and a refinement-stage model is used for enriching the details. Experiments demonstrate the superior performance of our method in terms of reconstruction accuracy and inference time compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-06-14T07:43:23Z)
• SAMVG: A Multi-stage Image Vectorization Model with the Segment-Anything Model [59.40189857428461]
  We propose a multi-stage model to vectorize images into SVG (Scalable Vector Graphics) Firstly, SAMVG uses general image segmentation provided by the Segment-Anything Model and uses a novel filtering method to identify the best dense segmentation map for the entire image. Secondly, SAMVG then identifies missing components and adds more detailed components to the SVG.
  arXiv  Detail & Related papers  (2023-11-09T11:11:56Z)
• Image Vectorization: a Review [4.258673477256579]
  Instead of generating vector images directly, you can first synthesize an image and then apply vectorization. In this paper, we focus specifically on machine learning-compatible vectorization methods.
  arXiv  Detail & Related papers  (2023-06-10T13:41:02Z)
• VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models [82.93345261434943]
  We show that a text-conditioned diffusion model trained on pixel representations of images can be used to generate SVG-exportable vector graphics. Inspired by recent text-to-3D work, we learn an SVG consistent with a caption using Score Distillation Sampling. Experiments show greater quality than prior work, and demonstrate a range of styles including pixel art and sketches.
  arXiv  Detail & Related papers  (2022-11-21T10:04:27Z)
• Towards Layer-wise Image Vectorization [57.26058135389497]
  We propose Layerwise Image Vectorization, namely LIVE, to convert images to SVGs and simultaneously maintain its image topology. Live generates compact forms with layer-wise structures that are semantically consistent with human perspective. Live initiates human editable SVGs for both designers and can be used in other applications.
  arXiv  Detail & Related papers  (2022-06-09T17:55:02Z)
• Recognizing Vector Graphics without Rasterization [36.31813939087549]
  We consider a different data format for images: vector graphics. In contrast to graphics which are widely used in image recognition, vector graphics can be scaled up or down into any resolution without aliasing or information loss. YOLaT builds multi-graphs to model the structural and spatial information in vector, and a dual-stream graph neural network is proposed to detect objects from the graph.
  arXiv  Detail & Related papers  (2021-11-05T06:16:17Z)
• SVG-Net: An SVG-based Trajectory Prediction Model [67.68864911674308]
  Anticipating motions of vehicles in a scene is an essential problem for safe autonomous driving systems. To this end, the comprehension of the scene's infrastructure is often the main clue for predicting future trajectories. Most of the proposed approaches represent the scene with averse averseized format and some of the more recent approaches leverage custom vectorized formats.
  arXiv  Detail & Related papers  (2021-10-07T18:00:08Z)
• DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation [217.86315551526235]
  We propose a novel hierarchical generative network, called DeepSVG, for complex SVG icons generation and manipulation. Our architecture effectively disentangles high-level shapes from the low-level commands that encode the shape itself. We demonstrate that our network learns to accurately reconstruct diverse vector graphics, and can serve as a powerful animation tool.
  arXiv  Detail & Related papers  (2020-07-22T09:36:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.