Character Generation through Self-Supervised Vectorization
- URL: http://arxiv.org/abs/2208.02012v1
- Date: Wed, 3 Aug 2022 12:31:55 GMT
- Title: Character Generation through Self-Supervised Vectorization
- Authors: Gokcen Gokceoglu and Emre Akbas
- Abstract summary: We present a drawing agent that operates on stroke-level representation of images.
When a 'draw' decision is made, the agent outputs a program indicating the stroke to be drawn.
We present successful results on all three generation tasks and the parsing task.
- Score: 9.36599317326032
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The prevalent approach in self-supervised image generation is to operate on
pixel level representations. While this approach can produce high quality
images, it cannot benefit from the simplicity and innate quality of
vectorization. Here we present a drawing agent that operates on stroke-level
representation of images. At each time step, the agent first assesses the
current canvas and decides whether to stop or keep drawing. When a 'draw'
decision is made, the agent outputs a program indicating the stroke to be
drawn. As a result, it produces a final raster image by drawing the strokes on
a canvas, using a minimal number of strokes and dynamically deciding when to
stop. We train our agent through reinforcement learning on MNIST and Omniglot
datasets for unconditional generation and parsing (reconstruction) tasks. We
utilize our parsing agent for exemplar generation and type conditioned concept
generation in Omniglot challenge without any further training. We present
successful results on all three generation tasks and the parsing task.
Crucially, we do not need any stroke-level or vector supervision; we only use
raster images for training.
Related papers
- Unsupervised Deep Learning Image Verification Method [0.0]
The proposed method achieves a relative improvement of 56% in terms of EER over the baseline system on Labeled Faces in the Wild dataset.
This has successfully narrowed down the performance gap between cosine and PLDA scoring systems.
arXiv Detail & Related papers (2023-12-22T02:52:54Z) - Image Vectorization: a Review [4.258673477256579]
Instead of generating vector images directly, you can first synthesize an image and then apply vectorization.
In this paper, we focus specifically on machine learning-compatible vectorization methods.
arXiv Detail & Related papers (2023-06-10T13:41:02Z) - MAGE: MAsked Generative Encoder to Unify Representation Learning and
Image Synthesis [33.46831766206675]
MAsked Generative (MAGE) is first framework to unify SOTA image generation and self-supervised representation learning.
Inspired by previous generative models, MAGE uses semantic tokens learned by a vector-quantized GAN at inputs and outputs.
On ImageNet-1K, a single MAGE ViT-L model obtains 9.10 FID in the task of class-unconditional image generation.
arXiv Detail & Related papers (2022-11-16T18:59:02Z) - Learning to Annotate Part Segmentation with Gradient Matching [58.100715754135685]
This paper focuses on tackling semi-supervised part segmentation tasks by generating high-quality images with a pre-trained GAN.
In particular, we formulate the annotator learning as a learning-to-learn problem.
We show that our method can learn annotators from a broad range of labelled images including real images, generated images, and even analytically rendered images.
arXiv Detail & Related papers (2022-11-06T01:29:22Z) - Drawing out of Distribution with Neuro-Symbolic Generative Models [49.79371715591122]
Drawing out of Distribution is a neuro-symbolic generative model of stroke-based drawing.
DooD operates directly on images, requires no supervision or expensive test-time inference.
We evaluate DooD on its ability to generalise across both data and tasks.
arXiv Detail & Related papers (2022-06-03T21:40:22Z) - Cluster-guided Image Synthesis with Unconditional Models [41.89334167530054]
This work focuses on controllable image generation by leveraging GANs that are well-trained in an unsupervised fashion.
By conditioning on the cluster assignments, the proposed method is able to control the semantic class of the generated image.
We showcase the efficacy of our approach on faces (CelebA-HQ and FFHQ), animals (Imagenet) and objects (LSUN) using different pre-trained generative models.
arXiv Detail & Related papers (2021-12-24T02:18:34Z) - EdiBERT, a generative model for image editing [12.605607949417033]
EdiBERT is a bi-directional transformer trained in the discrete latent space built by a vector-quantized auto-encoder.
We show that the resulting model matches state-of-the-art performances on a wide variety of tasks.
arXiv Detail & Related papers (2021-11-30T10:23:06Z) - Heredity-aware Child Face Image Generation with Latent Space
Disentanglement [96.92684978356425]
We propose a novel approach, called ChildGAN, to generate a child's image according to the images of parents with heredity prior.
The main idea is to disentangle the latent space of a pre-trained generation model and precisely control the face attributes of child images with clear semantics.
arXiv Detail & Related papers (2021-08-25T06:59:43Z) - Semantic Segmentation with Generative Models: Semi-Supervised Learning
and Strong Out-of-Domain Generalization [112.68171734288237]
We propose a novel framework for discriminative pixel-level tasks using a generative model of both images and labels.
We learn a generative adversarial network that captures the joint image-label distribution and is trained efficiently using a large set of unlabeled images.
We demonstrate strong in-domain performance compared to several baselines, and are the first to showcase extreme out-of-domain generalization.
arXiv Detail & Related papers (2021-04-12T21:41:25Z) - B\'ezierSketch: A generative model for scalable vector sketches [132.5223191478268]
We present B'ezierSketch, a novel generative model for fully vector sketches that are automatically scalable and high-resolution.
We first introduce a novel inverse graphics approach to stroke embedding that trains an encoder to embed each stroke to its best fit B'ezier curve.
This enables us to treat sketches as short sequences of paramaterized strokes and thus train a recurrent sketch generator with greater capacity for longer sketches.
arXiv Detail & Related papers (2020-07-04T21:30:52Z) - SketchyCOCO: Image Generation from Freehand Scene Sketches [71.85577739612579]
We introduce the first method for automatic image generation from scene-level freehand sketches.
Key contribution is an attribute vector bridged Geneversarative Adrial Network called EdgeGAN.
We have built a large-scale composite dataset called SketchyCOCO to support and evaluate the solution.
arXiv Detail & Related papers (2020-03-05T14:54:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.