Creative Painting with Latent Diffusion Models
- URL: http://arxiv.org/abs/2209.14697v2
- Date: Fri, 30 Sep 2022 03:08:34 GMT
- Title: Creative Painting with Latent Diffusion Models
- Authors: Xianchao Wu
- Abstract summary: latent diffusion models (LDMs) have achieved stable and high fertility image generation.
We focus on enhancing the creative painting ability of current LDMs in two directions, textual condition extension and model retraining with Wikiart dataset.
- Score: 1.4649095013539173
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Artistic painting has achieved significant progress during recent years.
Using an autoencoder to connect the original images with compressed latent
spaces and a cross attention enhanced U-Net as the backbone of diffusion,
latent diffusion models (LDMs) have achieved stable and high fertility image
generation. In this paper, we focus on enhancing the creative painting ability
of current LDMs in two directions, textual condition extension and model
retraining with Wikiart dataset. Through textual condition extension, users'
input prompts are expanded with rich contextual knowledge for deeper
understanding and explaining the prompts. Wikiart dataset contains 80K famous
artworks drawn during recent 400 years by more than 1,000 famous artists in
rich styles and genres. Through the retraining, we are able to ask these
artists to draw novel and creative painting on modern topics. Direct
comparisons with the original model show that the creativity and artistry are
enriched.
Related papers
- Stylebreeder: Exploring and Democratizing Artistic Styles through Text-to-Image Models [38.5896770458884]
We introduce textttSTYLEBREEDER, a comprehensive dataset of 6.8M images and 1.8M prompts generated by 95K users on Artbreeder.
By documenting unique, user-generated styles that transcend conventional categories like 'cyberpunk' or 'Picasso,' we explore the potential for unique, crowd-sourced styles.
Our research demonstrates the potential of text-to-image diffusion models to uncover and promote unique artistic expressions.
arXiv Detail & Related papers (2024-06-20T17:59:56Z) - Towards Highly Realistic Artistic Style Transfer via Stable Diffusion with Step-aware and Layer-aware Prompt [12.27693060663517]
Artistic style transfer aims to transfer the learned artistic style onto an arbitrary content image, generating artistic stylized images.
We propose a novel pre-trained diffusion-based artistic style transfer method, called LSAST.
Our proposed method can generate more highly realistic artistic stylized images than the state-of-the-art artistic style transfer methods.
arXiv Detail & Related papers (2024-04-17T15:28:53Z) - Rethinking Artistic Copyright Infringements in the Era of Text-to-Image Generative Models [47.19481598385283]
ArtSavant is a tool to determine the unique style of an artist by comparing it to a reference dataset of works from WikiArt.
We then perform a large-scale empirical study to provide quantitative insight on the prevalence of artistic style copying across 3 popular text-to-image generative models.
arXiv Detail & Related papers (2024-04-11T17:59:43Z) - CreativeSynth: Creative Blending and Synthesis of Visual Arts based on
Multimodal Diffusion [74.44273919041912]
Large-scale text-to-image generative models have made impressive strides, showcasing their ability to synthesize a vast array of high-quality images.
However, adapting these models for artistic image editing presents two significant challenges.
We build the innovative unified framework Creative Synth, which is based on a diffusion model with the ability to coordinate multimodal inputs.
arXiv Detail & Related papers (2024-01-25T10:42:09Z) - HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image Inpainting with Diffusion Models [59.01600111737628]
HD-Painter is a training free approach that accurately follows prompts and coherently scales to high resolution image inpainting.
To this end, we design the Prompt-Aware Introverted Attention (PAIntA) layer enhancing self-attention scores.
Our experiments demonstrate that HD-Painter surpasses existing state-of-the-art approaches quantitatively and qualitatively.
arXiv Detail & Related papers (2023-12-21T18:09:30Z) - Text-image guided Diffusion Model for generating Deepfake celebrity
interactions [50.37578424163951]
Diffusion models have recently demonstrated highly realistic visual content generation.
This paper devises and explores a novel method in that regard.
Our results show that with the devised scheme, it is possible to create fake visual content with alarming realism.
arXiv Detail & Related papers (2023-09-26T08:24:37Z) - Towards mapping the contemporary art world with ArtLM: an art-specific
NLP model [0.0]
We present a generic Natural Language Processing framework (called ArtLM) to discover the connections among contemporary artists based on their biographies.
With extensive experiments, we demonstrate that our ArtLM achieves 85.6% accuracy and 84.0% F1 score.
We also provide a visualisation and a qualitative analysis of the artist network built from ArtLM's outputs.
arXiv Detail & Related papers (2022-12-14T09:26:07Z) - Inversion-Based Style Transfer with Diffusion Models [78.93863016223858]
Previous arbitrary example-guided artistic image generation methods often fail to control shape changes or convey elements.
We propose an inversion-based style transfer method (InST), which can efficiently and accurately learn the key information of an image.
arXiv Detail & Related papers (2022-11-23T18:44:25Z) - Language Does More Than Describe: On The Lack Of Figurative Speech in
Text-To-Image Models [63.545146807810305]
Text-to-image diffusion models can generate high-quality pictures from textual input prompts.
These models have been trained using text data collected from content-based labelling protocols.
We characterise the sentimentality, objectiveness and degree of abstraction of publicly available text data used to train current text-to-image diffusion models.
arXiv Detail & Related papers (2022-10-19T14:20:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.