Scalable Font Reconstruction with Dual Latent Manifolds

• URL: http://arxiv.org/abs/2109.06627v1
• Date: Fri, 10 Sep 2021 20:37:43 GMT
• Title: Scalable Font Reconstruction with Dual Latent Manifolds
• Authors: Nikita Srivatsan, Si Wu, Jonathan T. Barron, Taylor Berg-Kirkpatrick
• Abstract summary: We propose a deep generative model that performs typography analysis and font reconstruction. Our approach enables us to massively scale up the number of character types we can effectively model. We evaluate on the task of font reconstruction over various datasets representing character types of many languages.
• Score: 55.29525824849242
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a deep generative model that performs typography analysis and font reconstruction by learning disentangled manifolds of both font style and character shape. Our approach enables us to massively scale up the number of character types we can effectively model compared to previous methods. Specifically, we infer separate latent variables representing character and font via a pair of inference networks which take as input sets of glyphs that either all share a character type, or belong to the same font. This design allows our model to generalize to characters that were not observed during training time, an important task in light of the relative sparsity of most fonts. We also put forward a new loss, adapted from prior work that measures likelihood using an adaptive distribution in a projected space, resulting in more natural images without requiring a discriminator. We evaluate on the task of font reconstruction over various datasets representing character types of many languages, and compare favorably to modern style transfer systems according to both automatic and manually-evaluated metrics.

Related papers

• Khattat: Enhancing Readability and Concept Representation of Semantic Typography [0.3994968615706021]
  semantic typography involves selecting an idea, choosing an appropriate font, and balancing creativity with readability. We introduce an end-to-end system that automates this process. Key feature is our OCR-based loss function, which enhances readability and enables simultaneous stylization of multiple characters.
  arXiv  Detail & Related papers  (2024-10-01T18:42:48Z)
• VQ-Font: Few-Shot Font Generation with Structure-Aware Enhancement and Quantization [52.870638830417]
  We propose a VQGAN-based framework (i.e., VQ-Font) to enhance glyph fidelity through token prior refinement and structure-aware enhancement. Specifically, we pre-train a VQGAN to encapsulate font token prior within a codebook. Subsequently, VQ-Font refines the synthesized glyphs with the codebook to eliminate the domain gap between synthesized and real-world strokes.
  arXiv  Detail & Related papers  (2023-08-27T06:32:20Z)
• Few-shot Font Generation by Learning Style Difference and Similarity [84.76381937516356]
  We propose a novel font generation approach by learning the Difference between different styles and the Similarity of the same style (DS-Font) Specifically, we propose a multi-layer style projector for style encoding and realize a distinctive style representation via our proposed Cluster-level Contrastive Style (CCS) loss.
  arXiv  Detail & Related papers  (2023-01-24T13:57:25Z)
• DGFont++: Robust Deformable Generative Networks for Unsupervised Font Generation [19.473023811252116]
  We propose a robust deformable generative network for unsupervised font generation (abbreviated as DGFont++) To distinguish different styles, we train our model with a multi-task discriminator, which ensures that each style can be discriminated independently. Experiments demonstrate that our model is able to generate character images of higher quality than state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-12-30T14:35:10Z)
• Diff-Font: Diffusion Model for Robust One-Shot Font Generation [110.45944936952309]
  We propose a novel one-shot font generation method based on a diffusion model, named Diff-Font. The proposed model aims to generate the entire font library by giving only one sample as the reference. The well-trained Diff-Font is not only robust to font gap and font variation, but also achieved promising performance on difficult character generation.
  arXiv  Detail & Related papers  (2022-12-12T13:51:50Z)
• Font Completion and Manipulation by Cycling Between Multi-Modality Representations [113.26243126754704]
  We innovate to explore the generation of font glyphs as 2D graphic objects with the graph as an intermediate representation. We formulate a cross-modality cycled image-to-image structure with a graph between an image encoder and an image. Our model generates improved results than both image-to-image baseline and previous state-of-the-art methods for glyph completion.
  arXiv  Detail & Related papers  (2021-08-30T02:43:29Z)
• Learning Perceptual Manifold of Fonts [7.395615703126767]
  We propose the perceptual manifold of fonts to visualize the perceptual adjustment in the latent space of a generative model of fonts. In contrast to the conventional user interface in our study, the proposed font-exploring user interface is efficient and helpful in the designated user preference.
  arXiv  Detail & Related papers  (2021-06-17T01:22:52Z)
• A Multi-Implicit Neural Representation for Fonts [79.6123184198301]
  font-specific discontinuities like edges and corners are difficult to represent using neural networks. We introduce textitmulti-implicits to represent fonts as a permutation-in set of learned implict functions, without losing features.
  arXiv  Detail & Related papers  (2021-06-12T21:40:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.