NIVeL: Neural Implicit Vector Layers for Text-to-Vector Generation

• URL: http://arxiv.org/abs/2405.15217v1
• Date: Fri, 24 May 2024 05:15:45 GMT
• Title: NIVeL: Neural Implicit Vector Layers for Text-to-Vector Generation
• Authors: Vikas Thamizharasan, Difan Liu, Matthew Fisher, Nanxuan Zhao, Evangelos Kalogerakis, Michal Lukac,
• Abstract summary: NIVeL reinterprets the problem on an alternative, intermediate domain which preserves the desirable properties of vector graphics. Based on our experiments, NIVeL produces text-to-vector graphics results of significantly better quality than the state-of-the-art.
• Score: 27.22029199085009
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The success of denoising diffusion models in representing rich data distributions over 2D raster images has prompted research on extending them to other data representations, such as vector graphics. Unfortunately due to their variable structure and scarcity of vector training data, directly applying diffusion models on this domain remains a challenging problem. Using workarounds like optimization via Score Distillation Sampling (SDS) is also fraught with difficulty, as vector representations are non trivial to directly optimize and tend to result in implausible geometries such as redundant or self-intersecting shapes. NIVeL addresses these challenges by reinterpreting the problem on an alternative, intermediate domain which preserves the desirable properties of vector graphics -- mainly sparsity of representation and resolution-independence. This alternative domain is based on neural implicit fields expressed in a set of decomposable, editable layers. Based on our experiments, NIVeL produces text-to-vector graphics results of significantly better quality than the state-of-the-art.

Related papers

• Segmentation-guided Layer-wise Image Vectorization with Gradient Fills [6.037332707968933]
  We propose a segmentation-guided vectorization framework to convert images into concise vector graphics with gradient fills. With the guidance of an embedded gradient-aware segmentation, our approach progressively appends gradient-filled B'ezier paths to the output.
  arXiv  Detail & Related papers  (2024-08-28T12:08:25Z)
• Optimize and Reduce: A Top-Down Approach for Image Vectorization [12.998637003026273]
  We propose Optimize & Reduce (O&R), a top-down approach to vectorization that is both fast and domain-agnostic. O&R aims to attain a compact representation of input images by iteratively optimizing B'ezier curve parameters. We demonstrate that our method is domain agnostic and outperforms existing works in both reconstruction and perceptual quality for a fixed number of shapes.
  arXiv  Detail & Related papers  (2023-12-18T16:41:03Z)
• FineMorphs: Affine-diffeomorphic sequences for regression [1.1421942894219896]
  The model states are optimally "reshaped" by diffeomorphisms generated by smooth vector fields during learning. Affine transformations and vector fields are optimized within an optimal control setting. The model can naturally reduce (or increase) dimensionality and adapt to large datasets via suboptimal vector fields.
  arXiv  Detail & Related papers  (2023-05-26T20:54:18Z)
• Polygonizer: An auto-regressive building delineator [12.693238093510072]
  We present an Image-to-Sequence model that allows for direct shape inference and is ready for vector-based out of the box. We demonstrate the model's performance in various ways, including perturbations to the image input that correspond to variations or artifacts commonly encountered in remote sensing applications.
  arXiv  Detail & Related papers  (2023-04-08T15:36:48Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Neural Vector Fields: Implicit Representation by Explicit Learning [63.337294707047036]
  We propose a novel 3D representation method, Neural Vector Fields (NVF) It not only adopts the explicit learning process to manipulate meshes directly, but also the implicit representation of unsigned distance functions (UDFs) Our method first predicts displacement queries towards the surface and models shapes as text reconstructions.
  arXiv  Detail & Related papers  (2023-03-08T02:36:09Z)
• Large-Margin Representation Learning for Texture Classification [67.94823375350433]
  This paper presents a novel approach combining convolutional layers (CLs) and large-margin metric learning for training supervised models on small datasets for texture classification. The experimental results on texture and histopathologic image datasets have shown that the proposed approach achieves competitive accuracy with lower computational cost and faster convergence when compared to equivalent CNNs.
  arXiv  Detail & Related papers  (2022-06-17T04:07:45Z)
• Latent Transformations via NeuralODEs for GAN-based Image Editing [25.272389610447856]
  We show that nonlinear latent code manipulations realized as flows of a trainable Neural ODE are beneficial for many practical non-face image domains. In particular, we investigate a large number of datasets with known attributes and demonstrate that certain attribute manipulations are challenging to obtain with linear shifts only.
  arXiv  Detail & Related papers  (2021-11-29T18:59:54Z)
• E(n) Equivariant Graph Neural Networks [86.75170631724548]
  This paper introduces a new model to learn graph neural networks equivariant to rotations, translations, reflections and permutations called E(n)-Equivariant Graph Neural Networks (EGNNs) In contrast with existing methods, our work does not require computationally expensive higher-order representations in intermediate layers while it still achieves competitive or better performance.
  arXiv  Detail & Related papers  (2021-02-19T10:25:33Z)
• Unrolling of Deep Graph Total Variation for Image Denoising [106.93258903150702]
  In this paper, we combine classical graph signal filtering with deep feature learning into a competitive hybrid design. We employ interpretable analytical low-pass graph filters and employ 80% fewer network parameters than state-of-the-art DL denoising scheme DnCNN.
  arXiv  Detail & Related papers  (2020-10-21T20:04:22Z)
• Learning Domain-invariant Graph for Adaptive Semi-supervised Domain Adaptation with Few Labeled Source Samples [65.55521019202557]
  Domain adaptation aims to generalize a model from a source domain to tackle tasks in a related but different target domain. Traditional domain adaptation algorithms assume that enough labeled data, which are treated as the prior knowledge are available in the source domain. We propose a Domain-invariant Graph Learning (DGL) approach for domain adaptation with only a few labeled source samples.
  arXiv  Detail & Related papers  (2020-08-21T08:13:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.