Text to artistic image generation

• URL: http://arxiv.org/abs/2205.02439v1
• Date: Thu, 5 May 2022 04:44:56 GMT
• Title: Text to artistic image generation
• Authors: Qinghe Tian (University of Waterloo), Jean-Claude Franchitti (New York University Courant Institute)
• Abstract summary: We create an end-to-end solution that can generate artistic images from text descriptions. Due to the lack of datasets with paired text description and artistic images, it is hard to directly train an algorithm which can create art based on text input.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Painting is one of the ways for people to express their ideas, but what if people with disabilities in hands want to paint? To tackle this challenge, we create an end-to-end solution that can generate artistic images from text descriptions. However, due to the lack of datasets with paired text description and artistic images, it is hard to directly train an algorithm which can create art based on text input. To address this issue, we split our task into three steps: (1) Generate a realistic image from a text description by using Dynamic Memory Generative Adversarial Network (arXiv:1904.01310), (2) Classify the image as a genre that exists in the WikiArt dataset using Resnet (arXiv: 1512.03385), (3) Select a style that is compatible with the genre and transfer it to the generated image by using neural artistic stylization network (arXiv:1705.06830).

Related papers

• CLII: Visual-Text Inpainting via Cross-Modal Predictive Interaction [23.683636588751753]
  State-of-the-art inpainting methods are mainly designed for nature images and cannot correctly recover text within scene text images. We identify the visual-text inpainting task to achieve high-quality scene text image restoration and text completion.
  arXiv  Detail & Related papers  (2024-07-23T06:12:19Z)
• Visual Text Generation in the Wild [67.37458807253064]
  We propose a visual text generator (termed SceneVTG) which can produce high-quality text images in the wild. The proposed SceneVTG significantly outperforms traditional rendering-based methods and recent diffusion-based methods in terms of fidelity and reasonability. The generated images provide superior utility for tasks involving text detection and text recognition.
  arXiv  Detail & Related papers  (2024-07-19T09:08:20Z)
• SketchDreamer: Interactive Text-Augmented Creative Sketch Ideation [111.2195741547517]
  We present a method to generate controlled sketches using a text-conditioned diffusion model trained on pixel representations of images. Our objective is to empower non-professional users to create sketches and, through a series of optimisation processes, transform a narrative into a storyboard.
  arXiv  Detail & Related papers  (2023-08-27T19:44:44Z)
• TextPainter: Multimodal Text Image Generation with Visual-harmony and Text-comprehension for Poster Design [50.8682912032406]
  This study introduces TextPainter, a novel multimodal approach to generate text images. TextPainter takes the global-local background image as a hint of style and guides the text image generation with visual harmony. We construct the PosterT80K dataset, consisting of about 80K posters annotated with sentence-level bounding boxes and text contents.
  arXiv  Detail & Related papers  (2023-08-09T06:59:29Z)
• Text-Guided Synthesis of Eulerian Cinemagraphs [81.20353774053768]
  We introduce Text2Cinemagraph, a fully automated method for creating cinemagraphs from text descriptions. We focus on cinemagraphs of fluid elements, such as flowing rivers, and drifting clouds, which exhibit continuous motion and repetitive textures.
  arXiv  Detail & Related papers  (2023-07-06T17:59:31Z)
• Paint it Black: Generating paintings from text descriptions [0.0]
  Two distinct tasks - generating photorealistic pictures from given text prompts and transferring the style of a painting to a real image to make it appear as though it were done by an artist, have been addressed many times, and several approaches have been proposed to accomplish them. In this paper, we have explored two distinct strategies and have integrated them together. First strategy is to generate photorealistic images and then apply style transfer and the second strategy is to train an image generation model on real images with captions and then fine-tune it on captioned paintings later.
  arXiv  Detail & Related papers  (2023-02-17T11:07:53Z)
• Exploring Stroke-Level Modifications for Scene Text Editing [86.33216648792964]
  Scene text editing (STE) aims to replace text with the desired one while preserving background and styles of the original text. Previous methods of editing the whole image have to learn different translation rules of background and text regions simultaneously. We propose a novel network by MOdifying Scene Text image at strokE Level (MOSTEL)
  arXiv  Detail & Related papers  (2022-12-05T02:10:59Z)
• Inversion-Based Style Transfer with Diffusion Models [78.93863016223858]
  Previous arbitrary example-guided artistic image generation methods often fail to control shape changes or convey elements. We propose an inversion-based style transfer method (InST), which can efficiently and accurately learn the key information of an image.
  arXiv  Detail & Related papers  (2022-11-23T18:44:25Z)
• Generative Art Using Neural Visual Grammars and Dual Encoders [25.100664361601112]
  A novel algorithm for producing generative art is described. It allows a user to input a text string, and which in a creative response to this string, outputs an image.
  arXiv  Detail & Related papers  (2021-05-01T04:21:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.