GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images

• URL: http://arxiv.org/abs/2003.02567v2
• Date: Tue, 21 Jul 2020 19:40:15 GMT
• Title: GANwriting: Content-Conditioned Generation of Styled Handwritten Word Images
• Authors: Lei Kang, Pau Riba, Yaxing Wang, Mar\c{c}al Rusi\~nol, Alicia Forn\'es and Mauricio Villegas
• Abstract summary: We take a step closer to producing realistic and varied artificially rendered handwritten words. We propose a novel method that is able to produce credible handwritten word images by conditioning the generative process with both calligraphic style features and textual content.
• Score: 10.183347908690504
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Although current image generation methods have reached impressive quality levels, they are still unable to produce plausible yet diverse images of handwritten words. On the contrary, when writing by hand, a great variability is observed across different writers, and even when analyzing words scribbled by the same individual, involuntary variations are conspicuous. In this work, we take a step closer to producing realistic and varied artificially rendered handwritten words. We propose a novel method that is able to produce credible handwritten word images by conditioning the generative process with both calligraphic style features and textual content. Our generator is guided by three complementary learning objectives: to produce realistic images, to imitate a certain handwriting style and to convey a specific textual content. Our model is unconstrained to any predefined vocabulary, being able to render whatever input word. Given a sample writer, it is also able to mimic its calligraphic features in a few-shot setup. We significantly advance over prior art and demonstrate with qualitative, quantitative and human-based evaluations the realistic aspect of our synthetically produced images.

Related papers

• Compositional Entailment Learning for Hyperbolic Vision-Language Models [54.41927525264365]
  We show how to fully leverage the innate hierarchical nature of hyperbolic embeddings by looking beyond individual image-text pairs. We propose Compositional Entailment Learning for hyperbolic vision-language models. Empirical evaluation on a hyperbolic vision-language model trained with millions of image-text pairs shows that the proposed compositional learning approach outperforms conventional Euclidean CLIP learning.
  arXiv  Detail & Related papers  (2024-10-09T14:12:50Z)
• RealCustom++: Representing Images as Real-Word for Real-Time Customization [80.04828124070418]
  Text-to-image customization aims to synthesize new images that align with both text semantics and subject appearance. Existing works follow the pseudo-word paradigm, which involves representing given subjects as pseudo-words. We propose a novel real-words paradigm termed RealCustom++ that instead represents subjects as non-conflict real words.
  arXiv  Detail & Related papers  (2024-08-19T07:15:44Z)
• DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Models [11.164432246850247]
  We introduce DreamStyler, a novel framework designed for artistic image synthesis. DreamStyler is proficient in both text-to-image synthesis and style transfer. With content and style guidance, DreamStyler exhibits flexibility to accommodate a range of style references.
  arXiv  Detail & Related papers  (2023-09-13T13:13:29Z)
• Text-Guided Synthesis of Eulerian Cinemagraphs [81.20353774053768]
  We introduce Text2Cinemagraph, a fully automated method for creating cinemagraphs from text descriptions. We focus on cinemagraphs of fluid elements, such as flowing rivers, and drifting clouds, which exhibit continuous motion and repetitive textures.
  arXiv  Detail & Related papers  (2023-07-06T17:59:31Z)
• WordStylist: Styled Verbatim Handwritten Text Generation with Latent Diffusion Models [8.334487584550185]
  We present a latent diffusion-based method for styled text-to-text-content-image generation on word-level. Our proposed method is able to generate realistic word image samples from different writer styles. We show that the proposed model produces samples that are aesthetically pleasing, help boosting text recognition performance, and get similar writer retrieval score as real data.
  arXiv  Detail & Related papers  (2023-03-29T10:19:26Z)
• Word-As-Image for Semantic Typography [41.380457098839926]
  A word-as-image is a semantic typography technique where a word illustration presents a visualization of the meaning of the word. We present a method to create word-as-image illustrations automatically.
  arXiv  Detail & Related papers  (2023-03-03T09:59:25Z)
• Inversion-Based Style Transfer with Diffusion Models [78.93863016223858]
  Previous arbitrary example-guided artistic image generation methods often fail to control shape changes or convey elements. We propose an inversion-based style transfer method (InST), which can efficiently and accurately learn the key information of an image.
  arXiv  Detail & Related papers  (2022-11-23T18:44:25Z)
• Language Does More Than Describe: On The Lack Of Figurative Speech in Text-To-Image Models [63.545146807810305]
  Text-to-image diffusion models can generate high-quality pictures from textual input prompts. These models have been trained using text data collected from content-based labelling protocols. We characterise the sentimentality, objectiveness and degree of abstraction of publicly available text data used to train current text-to-image diffusion models.
  arXiv  Detail & Related papers  (2022-10-19T14:20:05Z)
• An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion [60.05823240540769]
  Text-to-image models offer unprecedented freedom to guide creation through natural language. Here we present a simple approach that allows such creative freedom. We find evidence that a single word embedding is sufficient for capturing unique and varied concepts.
  arXiv  Detail & Related papers  (2022-08-02T17:50:36Z)
• Content and Style Aware Generation of Text-line Images for Handwriting Recognition [4.301658883577544]
  We propose a generative method for handwritten text-line images conditioned on both visual appearance and textual content. Our method is able to produce long text-line samples with diverse handwriting styles.
  arXiv  Detail & Related papers  (2022-04-12T05:52:03Z)
• SLOGAN: Handwriting Style Synthesis for Arbitrary-Length and Out-of-Vocabulary Text [35.83345711291558]
  We propose a novel method that can synthesize parameterized and controllable handwriting Styles for arbitrary-Length and Out-of-vocabulary text. We embed the text content by providing an easily obtainable printed style image, so that the diversity of the content can be flexibly achieved. Our method can synthesize words that are not included in the training vocabulary and with various new styles.
  arXiv  Detail & Related papers  (2022-02-23T12:13:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.