CCLAP: Controllable Chinese Landscape Painting Generation via Latent Diffusion Model

• URL: http://arxiv.org/abs/2304.04156v2
• Date: Sat, 22 Apr 2023 13:58:49 GMT
• Title: CCLAP: Controllable Chinese Landscape Painting Generation via Latent Diffusion Model
• Authors: Zhongqi Wang, Jie Zhang, Zhilong Ji, Jinfeng Bai, Shiguang Shan
• Abstract summary: controllable Chinese landscape painting generation method named CCLAP. Our method achieves state-of-the-art performance, especially in artfully-composed and artistic conception.
• Score: 54.74470985388726
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the development of deep generative models, recent years have seen great success of Chinese landscape painting generation. However, few works focus on controllable Chinese landscape painting generation due to the lack of data and limited modeling capabilities. In this work, we propose a controllable Chinese landscape painting generation method named CCLAP, which can generate painting with specific content and style based on Latent Diffusion Model. Specifically, it consists of two cascaded modules, i.e., content generator and style aggregator. The content generator module guarantees the content of generated paintings specific to the input text. While the style aggregator module is to generate paintings of a style corresponding to a reference image. Moreover, a new dataset of Chinese landscape paintings named CLAP is collected for comprehensive evaluation. Both the qualitative and quantitative results demonstrate that our method achieves state-of-the-art performance, especially in artfully-composed and artistic conception. Codes are available at https://github.com/Robin-WZQ/CCLAP.

Related papers

• Neural-Polyptych: Content Controllable Painting Recreation for Diverse Genres [30.83874057768352]
  We present a unified framework, Neural-Polyptych, to facilitate the creation of expansive, high-resolution paintings. We have designed a multi-scale GAN-based architecture to decompose the generation process into two parts. We validate our approach to diverse genres of both Eastern and Western paintings.
  arXiv  Detail & Related papers  (2024-09-29T12:46:00Z)
• Artistic Intelligence: A Diffusion-Based Framework for High-Fidelity Landscape Painting Synthesis [2.205829309604458]
  LPGen is a novel diffusion-based model specifically designed for landscape painting generation. LPGen introduces a decoupled cross-attention mechanism that independently processes structural and stylistic features. The model is pre-trained on a curated dataset of high-resolution landscape images, categorized by distinct artistic styles, and then fine-tuned to ensure detailed and consistent output.
  arXiv  Detail & Related papers  (2024-07-24T12:32:24Z)
• DLP-GAN: learning to draw modern Chinese landscape photos with generative adversarial network [20.74857981451259]
  Chinese landscape painting has a unique and artistic style, and its drawing technique is highly abstract in both the use of color and the realistic representation of objects. Previous methods focus on transferring from modern photos to ancient ink paintings, but little attention has been paid to translating landscape paintings into modern photos.
  arXiv  Detail & Related papers  (2024-03-06T04:46:03Z)
• Chinese Painting Style Transfer Using Deep Generative Models [0.0]
  Artistic style transfer aims to modify the style of the image while preserving its content. We will study and leverage different state-of-the-art deep generative models for Chinese painting style transfer.
  arXiv  Detail & Related papers  (2023-10-15T23:05:17Z)
• Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusion [50.59261592343479]
  We present Kandinsky1, a novel exploration of latent diffusion architecture. The proposed model is trained separately to map text embeddings to image embeddings of CLIP. We also deployed a user-friendly demo system that supports diverse generative modes such as text-to-image generation, image fusion, text and image fusion, image variations generation, and text-guided inpainting/outpainting.
  arXiv  Detail & Related papers  (2023-10-05T12:29:41Z)
• Unsupervised Compositional Concepts Discovery with Text-to-Image Generative Models [80.75258849913574]
  In this paper, we consider the inverse problem -- given a collection of different images, can we discover the generative concepts that represent each image? We present an unsupervised approach to discover generative concepts from a collection of images, disentangling different art styles in paintings, objects, and lighting from kitchen scenes, and discovering image classes given ImageNet images.
  arXiv  Detail & Related papers  (2023-06-08T17:02:15Z)
• BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing [73.74570290836152]
  BLIP-Diffusion is a new subject-driven image generation model that supports multimodal control. Unlike other subject-driven generation models, BLIP-Diffusion introduces a new multimodal encoder which is pre-trained to provide subject representation.
  arXiv  Detail & Related papers  (2023-05-24T04:51:04Z)
• Interactive Style Transfer: All is Your Palette [74.06681967115594]
  We propose a drawing-like interactive style transfer (IST) method, by which users can interactively create a harmonious-style image. Our IST method can serve as a brush, dip style from anywhere, and then paint to any region of the target content image.
  arXiv  Detail & Related papers  (2022-03-25T06:38:46Z)
• End-to-End Chinese Landscape Painting Creation Using Generative Adversarial Networks [0.0]
  We propose Sketch-And-Paint GAN (SAPGAN), the first model which generates Chinese landscape paintings from end to end, without conditional input. SAPGAN is composed of two GANs: SketchGAN for generation of edge maps, and PaintGAN for subsequent edge-to-painting translation. A 242-person Visual Turing Test study reveals that SAPGAN paintings are mistaken as human artwork with 55% frequency, significantly outperforming paintings from baseline GANs.
  arXiv  Detail & Related papers  (2020-11-11T05:20:42Z)
• SketchyCOCO: Image Generation from Freehand Scene Sketches [71.85577739612579]
  We introduce the first method for automatic image generation from scene-level freehand sketches. Key contribution is an attribute vector bridged Geneversarative Adrial Network called EdgeGAN. We have built a large-scale composite dataset called SketchyCOCO to support and evaluate the solution.
  arXiv  Detail & Related papers  (2020-03-05T14:54:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.