The Joy of Neural Painting

• URL: http://arxiv.org/abs/2111.10283v2
• Date: Mon, 22 Nov 2021 12:44:49 GMT
• Title: The Joy of Neural Painting
• Authors: Ernesto Diaz-Aviles (Libre AI) and Claudia Orellana-Rodriguez (Libre AI) and Beth Jochim (Libre AI)
• Abstract summary: We train a class of models that follows a GAN framework to generate brushstrokes, which are then composed to create paintings. To overcome GAN's limitations and to speed up the Neural Painter training, we applied Transfer Learning to the process reducing it from days to only hours.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Neural Painters is a class of models that follows a GAN framework to generate brushstrokes, which are then composed to create paintings. GANs are great generative models for AI Art but they are known to be notoriously difficult to train. To overcome GAN's limitations and to speed up the Neural Painter training, we applied Transfer Learning to the process reducing it from days to only hours, while achieving the same level of visual aesthetics in the final paintings generated. We report our approach and results in this work.

Related papers

• AttentionPainter: An Efficient and Adaptive Stroke Predictor for Scene Painting [82.54770866332456]
  Stroke-based Rendering (SBR) aims to decompose an input image into a sequence of parameterized strokes, which can be rendered into a painting that resembles the input image. We propose AttentionPainter, an efficient and adaptive model for single-step neural painting.
  arXiv  Detail & Related papers  (2024-10-21T18:36:45Z)
• Inverse Painting: Reconstructing The Painting Process [24.57538165449989]
  We formulate this as an autoregressive image generation problem, in which an initially blank "canvas" is iteratively updated. The model learns from real artists by training on many painting videos.
  arXiv  Detail & Related papers  (2024-09-30T17:56:52Z)
• ProcessPainter: Learn Painting Process from Sequence Data [27.9875429986135]
  The painting process of artists is inherently stepwise and varies significantly among different painters and styles. Traditional stroke-based rendering methods break down images into sequences of brushstrokes, yet they fall short of replicating the authentic processes of artists. We introduce ProcessPainter, a text-to-video model that is initially pre-trained on synthetic data and subsequently fine-tuned with a select set of artists' painting sequences.
  arXiv  Detail & Related papers  (2024-06-10T07:18:41Z)
• Stroke-based Neural Painting and Stylization with Dynamically Predicted Painting Region [66.75826549444909]
  Stroke-based rendering aims to recreate an image with a set of strokes. We propose Compositional Neural Painter, which predicts the painting region based on the current canvas. We extend our method to stroke-based style transfer with a novel differentiable distance transform loss.
  arXiv  Detail & Related papers  (2023-09-07T06:27:39Z)
• Interactive Neural Painting [66.9376011879115]
  This paper proposes the first approach for Interactive Neural Painting (NP) We propose I-Paint, a novel method based on a conditional transformer Variational AutoEncoder (VAE) architecture with a two-stage decoder. Our experiments show that our approach provides good stroke suggestions and compares favorably to the state of the art.
  arXiv  Detail & Related papers  (2023-07-31T07:02:00Z)
• Stroke-based Rendering: From Heuristics to Deep Learning [0.17188280334580194]
  Recent developments in deep learning methods help to bridge the gap between stroke-based paintings and pixel photo generation. We aim to provide a structured introduction and understanding of common challenges and approaches in stroke-based rendering algorithms.
  arXiv  Detail & Related papers  (2022-12-30T05:34:54Z)
• A domain adaptive deep learning solution for scanpath prediction of paintings [66.46953851227454]
  This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
  arXiv  Detail & Related papers  (2022-09-22T22:27:08Z)
• Perceptual Artifacts Localization for Inpainting [60.5659086595901]
  We propose a new learning task of automatic segmentation of inpainting perceptual artifacts. We train advanced segmentation networks on a dataset to reliably localize inpainting artifacts within inpainted images. We also propose a new evaluation metric called Perceptual Artifact Ratio (PAR), which is the ratio of objectionable inpainted regions to the entire inpainted area.
  arXiv  Detail & Related papers  (2022-08-05T18:50:51Z)
• Learning Prior Feature and Attention Enhanced Image Inpainting [63.21231753407192]
  This paper incorporates the pre-training based Masked AutoEncoder (MAE) into the inpainting model. We propose to use attention priors from MAE to make the inpainting model learn more long-distance dependencies between masked and unmasked regions.
  arXiv  Detail & Related papers  (2022-08-03T04:32:53Z)
• Intelli-Paint: Towards Developing Human-like Painting Agents [19.261822105543175]
  We propose a novel painting approach which learns to generate output canvases while exhibiting a more human-like painting style. Intelli-Paint consists of 1) a progressive layering strategy which allows the agent to first paint a natural background scene representation before adding in each of the foreground objects in a progressive fashion. We also introduce a novel sequential brushstroke guidance strategy which helps the painting agent to shift its attention between different image regions in a semantic-aware manner.
  arXiv  Detail & Related papers  (2021-12-16T14:56:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.