Few-shot Calligraphy Style Learning

• URL: http://arxiv.org/abs/2404.17199v1
• Date: Fri, 26 Apr 2024 07:17:09 GMT
• Title: Few-shot Calligraphy Style Learning
• Authors: Fangda Chen, Jiacheng Nie, Lichuan Jiang, Zhuoer Zeng,
• Abstract summary: "Presidifussion" is a novel approach to learning and replicating the unique style of calligraphy of President Xu. We introduce innovative techniques of font image conditioning and stroke information conditioning, enabling the model to capture the intricate structural elements of Chinese characters. This work not only presents a breakthrough in the digital preservation of calligraphic art but also sets a new standard for data-efficient generative modeling in the domain of cultural heritage digitization.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduced "Presidifussion," a novel approach to learning and replicating the unique style of calligraphy of President Xu, using a pretrained diffusion model adapted through a two-stage training process. Initially, our model is pretrained on a diverse dataset containing works from various calligraphers. This is followed by fine-tuning on a smaller, specialized dataset of President Xu's calligraphy, comprising just under 200 images. Our method introduces innovative techniques of font image conditioning and stroke information conditioning, enabling the model to capture the intricate structural elements of Chinese characters. The effectiveness of our approach is demonstrated through a comparison with traditional methods like zi2zi and CalliGAN, with our model achieving comparable performance using significantly smaller datasets and reduced computational resources. This work not only presents a breakthrough in the digital preservation of calligraphic art but also sets a new standard for data-efficient generative modeling in the domain of cultural heritage digitization.

Related papers

• Empowering Backbone Models for Visual Text Generation with Input Granularity Control and Glyph-Aware Training [68.41837295318152]
  Diffusion-based text-to-image models have demonstrated impressive achievements in diversity and aesthetics but struggle to generate images with visual texts. Existing backbone models have limitations such as misspelling, failing to generate texts, and lack of support for Chinese text. We propose a series of methods, aiming to empower backbone models to generate visual texts in English and Chinese.
  arXiv  Detail & Related papers  (2024-10-06T10:25:39Z)
• Ada-adapter:Fast Few-shot Style Personlization of Diffusion Model with Pre-trained Image Encoder [57.574544285878794]
  Ada-Adapter is a novel framework for few-shot style personalization of diffusion models. Our method enables efficient zero-shot style transfer utilizing a single reference image. We demonstrate the effectiveness of our approach on various artistic styles, including flat art, 3D rendering, and logo design.
  arXiv  Detail & Related papers  (2024-07-08T02:00:17Z)
• Reinforcing Pre-trained Models Using Counterfactual Images [54.26310919385808]
  This paper proposes a novel framework to reinforce classification models using language-guided generated counterfactual images. We identify model weaknesses by testing the model using the counterfactual image dataset. We employ the counterfactual images as an augmented dataset to fine-tune and reinforce the classification model.
  arXiv  Detail & Related papers  (2024-06-19T08:07:14Z)
• Information Theoretic Text-to-Image Alignment [49.396917351264655]
  We present a novel method that relies on an information-theoretic alignment measure to steer image generation. Our method is on-par or superior to the state-of-the-art, yet requires nothing but a pre-trained denoising network to estimate MI.
  arXiv  Detail & Related papers  (2024-05-31T12:20:02Z)
• MuseumMaker: Continual Style Customization without Catastrophic Forgetting [50.12727620780213]
  We propose MuseumMaker, a method that enables the synthesis of images by following a set of customized styles in a never-end manner. When facing with a new customization style, we develop a style distillation loss module to extract and learn the styles of the training data for new image generation. It can minimize the learning biases caused by content of new training images, and address the catastrophic overfitting issue induced by few-shot images.
  arXiv  Detail & Related papers  (2024-04-25T13:51:38Z)
• ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank [9.99530386586636]
  Artistic style transfer aims to repaint the content image with the learned artistic style. Existing artistic style transfer methods can be divided into two categories: small model-based approaches and pre-trained large-scale model-based approaches. We propose ArtBank, a novel artistic style transfer framework, to generate highly realistic stylized images.
  arXiv  Detail & Related papers  (2023-12-11T05:53:40Z)
• Customize StyleGAN with One Hand Sketch [0.0]
  We propose a framework to control StyleGAN imagery with a single user sketch. We learn a conditional distribution in the latent space of a pre-trained StyleGAN model via energy-based learning. Our model can generate multi-modal images semantically aligned with the input sketch.
  arXiv  Detail & Related papers  (2023-10-29T09:32:33Z)
• Self-Supervised Representation Learning for Online Handwriting Text Classification [0.8594140167290099]
  We propose the novel Part of Stroke Masking (POSM) as a pretext task for pretraining models to extract informative representations from the online handwriting of individuals in English and Chinese languages. To evaluate the quality of the extracted representations, we use both intrinsic and extrinsic evaluation methods. The pretrained models are fine-tuned to achieve state-of-the-art results in tasks such as writer identification, gender classification, and handedness classification.
  arXiv  Detail & Related papers  (2023-10-10T14:07:49Z)
• LiT Tuned Models for Efficient Species Detection [22.3395465641384]
  Our paper introduces a simple methodology for adapting any fine-grained image classification dataset for distributed vision-language pretraining. We implement this methodology on the challenging iNaturalist-2021 dataset, comprised of approximately 2.7 million images of macro-organisms across 10,000 classes. Our model (trained using a new method called locked-image text tuning) uses a pre-trained, frozen vision representation, proving that language alignment alone can attain strong transfer learning performance.
  arXiv  Detail & Related papers  (2023-02-12T20:36:55Z)
• PaCaNet: A Study on CycleGAN with Transfer Learning for Diversifying Fused Chinese Painting and Calligraphy [8.724826658340415]
  PaCaNet is a CycleGAN-based pipeline for producing novel artworks that fuse two different art types, traditional Chinese painting and calligraphy. Our approach creates a unique aesthetic experience rooted in the origination of Chinese hieroglyph characters.
  arXiv  Detail & Related papers  (2023-01-30T17:22:10Z)
• Shifted Diffusion for Text-to-image Generation [65.53758187995744]
  Corgi is based on our proposed shifted diffusion model, which achieves better image embedding generation from input text. Corgi also achieves new state-of-the-art results across different datasets on downstream language-free text-to-image generation tasks.
  arXiv  Detail & Related papers  (2022-11-24T03:25:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.