A Framework and Dataset for Abstract Art Generation via CalligraphyGAN
- URL: http://arxiv.org/abs/2012.00744v1
- Date: Wed, 2 Dec 2020 16:24:20 GMT
- Title: A Framework and Dataset for Abstract Art Generation via CalligraphyGAN
- Authors: Jinggang Zhuo, Ling Fan, Harry Jiannan Wang
- Abstract summary: We present a creative framework based on Conditional Generative Adversarial Networks and Contextual Neural Language Model to generate abstract artworks.
Our work is inspired by Chinese calligraphy, which is a unique form of visual art where the character itself is an aesthetic painting.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: With the advancement of deep learning, artificial intelligence (AI) has made
many breakthroughs in recent years and achieved superhuman performance in
various tasks such as object detection, reading comprehension, and video games.
Generative Modeling, such as various Generative Adversarial Networks (GAN)
models, has been applied to generate paintings and music. Research in Natural
Language Processing (NLP) also had a leap forward in 2018 since the release of
the pre-trained contextual neural language models such as BERT and recently
released GPT3. Despite the exciting AI applications aforementioned, AI is still
significantly lagging behind humans in creativity, which is often considered
the ultimate moonshot for AI. Our work is inspired by Chinese calligraphy,
which is a unique form of visual art where the character itself is an aesthetic
painting. We also draw inspirations from paintings of the Abstract
Expressionist movement in the 1940s and 1950s, such as the work by American
painter Franz Kline. In this paper, we present a creative framework based on
Conditional Generative Adversarial Networks and Contextual Neural Language
Model to generate abstract artworks that have intrinsic meaning and aesthetic
value, which is different from the existing work, such as image captioning and
text-to-image generation, where the texts are the descriptions of the images.
In addition, we have publicly released a Chinese calligraphy image dataset and
demonstrate our framework using a prototype system and a user study.
Related papers
- Autoregressive Models in Vision: A Survey [119.23742136065307]
This survey comprehensively examines the literature on autoregressive models applied to vision.
We divide visual autoregressive models into three general sub-categories, including pixel-based, token-based, and scale-based models.
We present a multi-faceted categorization of autoregressive models in computer vision, including image generation, video generation, 3D generation, and multi-modal generation.
arXiv Detail & Related papers (2024-11-08T17:15:12Z) - No Longer Trending on Artstation: Prompt Analysis of Generative AI Art [7.64671395172401]
We collect and analyse over 3 million prompts and the images they generate.
Our study shows that prompting focuses largely on surface aesthetics, reinforcing cultural norms, popular conventional representations and imagery.
arXiv Detail & Related papers (2024-01-24T08:03:13Z) - Artistic Strategies to Guide Neural Networks [0.0]
This paper explores the potentials and limits of current AI technology, in the context of image, text, form and translation of semiotic spaces.
In a relatively short time, the generation of high-resolution images and 3D objects has been achieved.
Yet again, we see how artworks act as catalysts for technology development.
arXiv Detail & Related papers (2023-07-06T22:57:10Z) - Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion
Models [70.86603627188519]
We focus on a novel, yet challenging task of generating a coherent image sequence based on a given storyline, denoted as open-ended visual storytelling.
We propose a learning-based auto-regressive image generation model, termed as StoryGen, with a novel vision-language context module.
We show StoryGen can generalize to unseen characters without any optimization, and generate image sequences with coherent content and consistent character.
arXiv Detail & Related papers (2023-06-01T17:58:50Z) - AIwriting: Relations Between Image Generation and Digital Writing [0.0]
During 2022, AI text generation systems such as GPT-3 and AI text-to-image generation systems such as DALL-E 2 made exponential leaps forward.
In this panel a group of electronic literature authors and theorists consider new oppor-tunities for human creativity presented by these systems.
arXiv Detail & Related papers (2023-05-18T09:23:05Z) - Language Does More Than Describe: On The Lack Of Figurative Speech in
Text-To-Image Models [63.545146807810305]
Text-to-image diffusion models can generate high-quality pictures from textual input prompts.
These models have been trained using text data collected from content-based labelling protocols.
We characterise the sentimentality, objectiveness and degree of abstraction of publicly available text data used to train current text-to-image diffusion models.
arXiv Detail & Related papers (2022-10-19T14:20:05Z) - Visualize Before You Write: Imagination-Guided Open-Ended Text
Generation [68.96699389728964]
We propose iNLG that uses machine-generated images to guide language models in open-ended text generation.
Experiments and analyses demonstrate the effectiveness of iNLG on open-ended text generation tasks.
arXiv Detail & Related papers (2022-10-07T18:01:09Z) - On Advances in Text Generation from Images Beyond Captioning: A Case
Study in Self-Rationalization [89.94078728495423]
We show that recent advances in each modality, CLIP image representations and scaling of language models, do not consistently improve multimodal self-rationalization of tasks with multimodal inputs.
Our findings call for a backbone modelling approach that can be built on to advance text generation from images and text beyond image captioning.
arXiv Detail & Related papers (2022-05-24T00:52:40Z) - A Taxonomy of Prompt Modifiers for Text-To-Image Generation [6.903929927172919]
This paper identifies six types of prompt modifier used by practitioners in the online community based on a 3-month ethnography study.
The novel taxonomy of prompt modifier provides researchers a conceptual starting point for investigating the practice of text-to-image generation.
We discuss research opportunities of this novel creative practice in the field of Human-Computer Interaction.
arXiv Detail & Related papers (2022-04-20T06:15:50Z) - GAN Computers Generate Arts? A Survey on Visual Arts, Music, and
Literary Text Generation using Generative Adversarial Network [0.0]
This survey takes a look at the recent works using GANs for generating visual arts, music, and literary text.
Some of the key challenges in art generation using GANs are highlighted along with recommendations for future work.
arXiv Detail & Related papers (2021-08-09T07:59:04Z) - State of the Art on Neural Rendering [141.22760314536438]
We focus on approaches that combine classic computer graphics techniques with deep generative models to obtain controllable and photo-realistic outputs.
This report is focused on the many important use cases for the described algorithms such as novel view synthesis, semantic photo manipulation, facial and body reenactment, relighting, free-viewpoint video, and the creation of photo-realistic avatars for virtual and augmented reality telepresence.
arXiv Detail & Related papers (2020-04-08T04:36:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.